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ABSTRACT 

3-D  reconstruction  technique  plays  an  important  role  in  the  applications  for  3-D  data  acquisition,  such  as  medical 
diagnosis,  animation  and  virtual  reality.  Moreover,  the  3-D  triangulation  process  is  one  of  the  most  important  parts  while 
reconstructing  the  smooth  surface  of  a 3D  object.  The  essential  of  3-D  triangulation  is  to  find  the  intersection  of  the  rays 
emitting  from  the  correlated  points  on  each  image  pair,  but  the  emitting  rays  always  don’t  intersect  with  each  other  owing  to 
the  error  in  process.  So  the  obtained  3-D  point  is  an  approximated  value  and  makes  the  reconstructed  surface  uneven.  In  the 
paper,  we  propose  an  triangulation  enhancement  method,  which  reduces  the  perturbation  in  the  reconstructed  data  and  filter 
out  the  error  caused  by  spurious  vectors  in  the  process  of  correlation. 

Keywords:  3-D  geometry,  correlation  vector  field,  3-D  triangulation,  multiply  factor,  3-D  stereo  imaging  system. 

1.  INTRODUCTION 

The  reconstruction  of  3D  object  from  stereo  images  has  been  widely  investigated  for  various  applications,  such  as 
medical  diagnosis,  virtual  reality,  and  animation,  etc.  In  the  processes  of  reconstruction,  3D  data  are  mainly  determined  by 
data  correlation  between  two  image  frames  and  3D  triangulation  from  two  correlated  points.  The  correlation  process  is 
adopted  to  calculate  the  correspondence  of  data  points  of  two  image  frames  to  generate  correlated  vector  fields.  According 
to  the  relation  provided  by  the  vector  field,  the  triangulation  process  will  produce  3D  coordinates  of  an  object.  The  concept 
of  the  technique  is  to  restore  the  3-D  coordinates  of  an  object  by  using  the  point  correlation  on  two  image  frames  and  the 
extrinsic/intrinsic  parameters[l].  The  extrinsic  parameters  depict  the  location  and  orientation  of  the  camera  reference  frame 
with  respect  to  a world  reference  frame,  whereas  the  intrinsic  parameters  describe  the  connection  of  the  pixel  coordinates  of 
an  image  point  and  the  corresponding  coordinates  in  the  camera  reference  frame.  Whether  the  parameters  are  known  or  not, 
Trucco[l]  classified  the  reconstruction  problems  into  three  cases:  (1)  intrinsic  and  extrinsic  parameters  are  known,  (2)  only 
intrinsic  parameters  are  known,  and  (3)  no  information  on  parameters.  However,  due  to  the  error  of  correlation  and 
calibration  processes,  it  is  still  impossible  to  get  a totally  correct  solution  even  if  in  case  of  (1).  The  phenomena  of  case(l) 
are  the  fundamental  issue  we  should  cope  with.  Our  research  aims  to  find  an  enhanced  triangulation  method,  which  could 
filter  out  the  spurious  vectors  in  the  correlation  vector  field  and  reduce  the  3-D  error. 

In  Section  2,  we  describe  the  3-D  stereo  imaging  system  based  on  the  direct  triangulation  and  an  enhancement 
triangulation  is  proposed  in  section  3.  Section  4 illustrates  the  experimental  results  and  finally  section  5 gives  the 
conclusions. 


2.  3-D  STEREO  IMAGING  SYSTEM  BASED  ON  DIRECT  TRIANGULATION 

2,1  Stereo  imaging  system 

In  the  paper,  a 3-D  stereo  imaging  system  with  laser  speckle  projection  is  proposed  to  capture  the  stereo  images  and 
reconstruct  the  shape  of  a 3-D  object(see  Figure  1). 
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Figure  1 The  3-D  stereo  imaging  system  with  laser  speckle  projection. 


In  the  aspect  of  processing,  a 3-D  stereo  imaging  system  contains  three  major  modules:  (1)  projection  module,  (2) 
correlation  module  and  (3)  triangulation  module.  In  the  projection  module,  the  randomized  patterns  are  produced  by  letting 
laser  beam  pass  through  a diffuser  and  projected  on  the  object  surface,  which  provides  the  reference  information  for  the 
correlation  module.  There  are  some  criteria  about  how  to  select  an  appropriate  speckle  pattern  for  better  correlation 
result([2]).  Then,  in  correlation  module,  we  compute  the  correlation  by  using  the  relation  between  the  two  image  frames  and 
build  the  correlation  vector  field,  which  describes  the  correlation  of  the  points  on  left  and  right  image  frames.  In  order  to  get 
high  speed  processing,  we  adopt  the  method  of  compressed  image  correlation  proposed  by  Hart([3],  [4]  and[5]).  The  method 
has  been  successfully  applied  to  track  the  flow  vectors  in  PIV  images. 

Finally,  the  triangulation  module  is  adopted  to  calculate  the  3-D  coordinates  from  the  vector  field  and  reconstructs  the 
3-D  object.  The  essential  of  triangulation  module  is  to  find  the  intersection  of  the  rays  emitting  from  the  pair  of  correlated 
points.  In  ideal  case,  the  3-D  rays  corresponding  to  the  correlation  points  on  the  image  frames  should  intersect  with  each 
other  at  a point,  P”(see  Figure  2).  Flowever,  in  practical  situation,  the  two  rays  will  be  skew  in  3-D  space  without 
intersection.  The  triangulation  error  comes  mainly  from  two  error  sources:  (1)  correlation  error  and  (2)  CCD  calibration 
error.  We  identify  three  conditions  by  considering  the  two  error  sources: 

• The  correlation  and  CCD  calibration  is  perfect  and  error  free. 

The  de-projection  rays(L  and  R”)  intersect  at  point  P”(see  Figure  2). 

• The  correlation  is  perfect  but  with  CCD  calibration  error. 

The  correlation  pair  is  still  (PL  ,P’R),  however,  the  de-projection  ray  R”  corresponding  to  P’R  moves  to  R’  because 
of  the  calibration  errorfsee  Figure  2).  The  triangulation  point  P’  is  estimated  from  the  shortest  path  between  both  rays. 

• Both  correlation  and  CCD  calibration  induce  error. 

This  is  the  most  common  condition.  The  correlated  point  in  the  right  image  frame  shifts  rom  P’R  to  PR,  moreover, 
the  de-projected  rays  is  Rand  the  triangulation  point  becomes  P(see  Figure  2). 

In  the  paper,  our  system  is  based  on  the  third  condition  since  the  the  results  of  the  correlation  and  calibration  processes 
should  bring  error  in  real  world.  The  computation  of  triangulation  P is  as  following: 

(1)  Let  ray  L be  a P,  and  ray  R be  T + j3 RT PR  , then  the  vector  w,  which  is  orthogonal  to  both  rays  L and  R,  is  described 

as  w~PL  xRtPr.  (Eq.l) 

(2)  Solve  the  equation 

aPL-  PRt  +yw=T  ■ (Bq.2) 

Solving  Eq.2  will  determine  the  end  points  of  the  shortest  path(s)  between  two  rays,  and  the  triangulation  point  P is  the 
midpoint  of  the  path  s. 

The  correlation  error  dominates  the  3-D  triangulation  error  because  the  2-D  errors  are  enhanced  while  2-D  points  in  image 
frames  are  converted  to  3-D  rays.  In  section  2.2,  the  3-D  error  caused  by  correlation  error  will  be  analyzed  and  discussed. 
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2.2  Relation  between  2-D  shift  and  3-D  displacement 

Many  issuses  affect  the  correctness  of  correlation,  for  example,  the  quality  of  the  spackle  projection,  the  slope  of  object 
surface,  the  CCD  inherent  disturbance,  etc.  We  make  an  experiment  to  investigate  and  measure  the  effect  on  3-D  space 
when  the  correctness  of  corelation  vector  is  corrupted  by  the  issues: 

Step  1.  Take  a set  of  CCD  parameters  obtained  in  the  calibration  process. 

Step  2.  Grid  the  left  image  frame  and  use  the  position  of  cross-intersection  as  the  left  point  set(SL). 

Step  3.  Assume  the  3-D  surface  is  z=0  and  de-project  SL  the  3-D  point  set(S3.D)  by  using  the  CCD  parameters. 

Step  4.  Project  S3.D  to  right  image  frame  and  get  the  right  point  set(SR). 

Step  5.  Shift  the  right  point  set  in  x-  or  y-  direction  as  the  correlation  error  to  calculate  the  influence  on  the  coordinates  of 
the  triangulation  point. 

Figure  3 shows  the  relationship  between  mean/STD  and  x/y  shift.  Figure  3(a)  illustrates  that  the  3-D  displacement  order  is 
around  10 3 in  the  shift  of  y-direction  and  Figure  3(b)  represents  that  the  shift  in  x-direction  has  much  displacement,  the 
order  is  around  10  . Besiedes,  since  the  STD  is  low  in  both,  it  depicts  that  the  point  on  the  image  frame  with  the  same  shift 
has  similar  effect  on  3-D  displacement. 
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Figure  3 Mean  and  STD  on  Z-direction  direction. 
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From  the  observation,  we  conclude  that  the  depth-shift  ratio  is  0.8-1. 1 mm/pixel  in  x-direction,  in  other  words,  the  z 
coordinate  will  shift  about  1mm  if  the  correction  vector  has  pixel-level  error.  Of  course,  the  depth  resolution  is  dependent 
on  the  photogeometry  of  the  stereo  system,  e.g.  the  distance  between  CCDs.  If  the  distance  between  CCDs  increases,  then 
the  depth  resolution  will  increase.  But  it  is  unreasonable  to  enlarge  the  distance  unlimitedly.  In  our  system,  the  distance  is 
approximately  60  mm. 


3.  THE  ENHANCEMENT  TRIANGULATION 

The  system  based  on  direct  triangulation  method  faces  two  major  problems: 

(1)  The  z-coordinates  of  the  reconstruction  surface  are  sensitive  to  the  correlation  error. 

(2)  Some  spurious  vector  still  survive  after  Correlation  Error  Correction(CEC)  in  the  correlation  process. 

Problem  (1)  will  be  discussed  in  section  3.1  and  section  3.2  gives  an  solution  to  problem(2). 

3.1  Vector  Mapping  Method 

Since  the  stereo  system  has  two  CCDs,  the  direct  triangulation  will  enlarge  the  error  while  the  2-D  correlated  pair 
is  converted  to  3-D  space  rays.  Little  shift  of  correlation  vectors  induces  larger  3-D  depth  error.  A vector  mapping  method  is 
proposed  to  directly  convert  the  2-D  pixel  coordinate  (x,  y)  to  3-D  space  coordinate  (X,  Y,  Z)  by  using  three  scale 
factors(Kx,  Ky,  Kz).  This  will  reduce  the  3-D  perturbation  caused  ion  from  . The  central  part  of  the  method  is  to  compute  the 
scale  factors. 

Figure  4 illustrates  the  computation  of  Kx  and  Kv.  Select  a observation  space  where  the  object  will  be  located,  then 
calculate  the  scale  factors  Kx  and  KY  by  using  the  relationship  between  (X2-X,,  YrY,)  and  (x2-x,,  y2-yi). 


Figure  4 Computation  of  Kx  and  KY. 


The  length  of  the  correlation  vector  is  variant  when  a 3-D  point  moves  with  different  distance  from  CCDs. 
According  to  the  relation  of  the  vector  mean  length  related  to  the  varied  distance  between  the  3-D  point  and  CCDs,  Kz  is 
computed  from  the  ratio  of  the  mean  length  and  distance.  Surely,  the  value  of  Kz  should  vary  with  the  distance,  however,  Kz 
will  approximate  a constant  if  the  z-depth  of  the  point  is  in  a certain  range.  Here,  we  use  a template  to  calibrate  Kz  (see 
Figure  5).  Firstly,  we  place  the  template  at  the  position  at  Z=0,  and  compute  the  mean  length(/en('K0jj  of  the  vectors  in 
correlation  vector  field.  Then  we  move  the  template  to  another  position  Z = t,  also  compute  the  mean  \sx\g\h(len(V )) . Kz 
related  to  position  Z = t is  calculated  from  t / (lenfVJ-  len(Vg)). 
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Figure  5 Calibration  of  Kz. 


The  relationship  between  2-D  pixel  coordinate  (x,y)  and  3-D  space  coordinate  (X,  Y Z)  is  formulated  as 
X = Xs  + kx  * x 

Y = Ys+ky*y  (Eq.  3) 

. Z = Kz*(len(Vx)-len(Vx0 )) 
where  Xs  and  Ys  are  the  initial  position  of  3-D  space. 


3.2  Distance  Map 


The  3-D  rays  of  the  correlated  point  pair  often  doesn’t  intersect  with  each  other  because  of  the  corelation  and 
calibration  errors.  The  distance  between  two  rays  gives  a direction  about  how  to  filter  the  spurious  vectors.  Figure  6 shows 
the  experiment  of  computing  the  shape  object  of  half-sphere. 


(c)  Correlation  vector  field. 


Figure  6 Correlation  vector  field  and  distance  map  of  the  half  sphere. 
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Figure  6(a)  and  3(b)  are  the  left  and  right  speckle  images  of  the  half  sphere.  The  correlation  vector  field 
corresponding  to  the  two  images  is  shown  in  Figure  6(c).  According  to  the  smoothness  criteria,  we  find  that  maybe  some 
spurious  vectors  occur  in  region  1 to  region4,  where  the  vectors  are  obviously  in  the  different  direction.  However,  the 
vectors  could  possibly  locate  on  the  edge  or  peak  of  the  object.  We  cannot  make  sure  that  they  are  exactly  spurious  vectors 
or  not  even  if  they  pass  the  CEC  process  in  the  correlation  module.  But  from  the  distance  map(see  Figure  6(d)) 
corresponding  to  the  vector  field,  the  related  areas  have  significant  high  distance.  In  other  words,  there  is  a proportional 
relationship  between  distance  map  and  correlation  vector  field,  i.e.,  some  position  with  large  value  in  distance  map 
represents  the  corresponding  vector  is  spurious  vector.  This  phenomenon  provides  a useful  indication  to  identify  the 
spurious  vectors.  We  can  filter  out  the  spurious  vector  by  observing  the  corresponding  distance  in  the  map. 

3.3  Procedures  Related  to  Enhancement  Triangulation 

Figure  7 illustrates  the  related  procedures  of  triangulation  enhancement.  The  method  includes  the  generation  of 
distance  map  exploited  to  filter  out  the  spurious  vectors  and  the  calibration  of  multiple  factors  (Kx,  KY,  Kz)  applied  to 
compute  the  3-D  coordinates  of  object. 


4.  EXPERIMENTAL  RESULTS 

In  the  section,  we  compare  the  experimental  results  from  direct  triangulation  and  triangulation  enhancement 
separately.  The  distance  between  two  CCDs  is  65  mm  and  the  object  is  550  mm  from  the  CCDs.  Figure  8 displays  the  left 
and  right  speckle  projection  images  captured  from  the  stereo  system. 
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Figure  8 Left  and  right  images  of  Venus  Head. 


Figure  9 is  the  result  by  using  the  direct  triangulation  method  and  Figure  10  shows  the  reconstructed  surface  through  the 
method  of  triangulation  enhancement. 


Figure  9 The  reconstructed  object  from  direct  triangulation. 


Figure  10  The  reconstructed  object  from  triangulation  enhancement. 


Compared  Figure  9 with  Figure  10,  the  surface  points  with  large  error  are  filtered  after  the  enhancement,  especially  on  the 
right  cheek.  Besides,  the  surface  in  Figure  10  is  smoother  than  Figure  9,  and  the  difference  is  particularly  obvious  on  the  left 
cheek.  The  reason  is  that  the  histogram  of  the  spackled  image  is  not  very  the  same(see  Figure  8),  so  the  correlation  errors  in 
the  part  of  left  cheek  are  larger  than  other  area  of  the  face.  The  direct  triangulation  will  enlarge  the  influence  of  the 
correlation  error  while  converting  2-D  image  pairs  to  3-D  points,  however,  the  triangulation  enhancement  has  smaller 
perturbation  in  3-D  surface  since  it  just  reflects  the  correlation  error. 


5.  CONCLUSIONS 

In  the  paper,  we  propose  a triangulation  enhancement  in  a stereo  imaging  system.  The  proposed  method  gets 
smoother  surface  than  direct  triangulation  by  reducing  3-d  error  through  vector  mapping  method,  moreover,  it  filters  out  the 
spurious  errors  by  exploiting  the  distance  map.  By  observing  the  experimental  results,  we  conclude  that  over  85%  of  the  3- 
D errors  are  influenced  by  correlation  error.  Therefore,  the  correlation  algorithm  on  sub-pixel  level  should  be  improved  to 
increase  the  correlation  accuracy.  Furthermore,  the  depth  resolution  is  another  important  consideration;  in  the  other  words, 
the  reconstructed  surface  will  achieve  better  quality  if  the  depth-shift  ratio  becomes  lower.  Our  future  work  will  focus  on 
how  to  improve  the  accuracy  of  the  correlation  algorithm  and  increase  the  depth  resolution. 
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